DOCUMENT RESUME 



IR 016 031 

McElveen, Lee K. ; Roberts, Sharon P. 
Telelearning: A Second Look. 1990-1991, 1991-1992. 
25 Sep 92 
20p. 

Reports - Descriptive (141) — Reports - 
Evaluative/Feasibility (142) 

MFOl/PCOl Plus Postage. 

Comparative Analysis; ^Curriculum Evaluation; 
^Distance Education; '^Evaluation Methods; Fine Arts; 
Formative Evaluation; High Schools; High School 
Students; Mathematics Education; Outcomes of 
Education; Proctoring; Science Education; Secondary 
School Teachers; '^Student Attitudes; Teacher 
Effectiveness; ''^Teaching Methods; Technological 
Advancement 

Audiographics; '''Louisiana; Student Surveys; Teacher 
Surveys; ^Telelearning Project LA 



Telelearning, Project Outreach, a Louisiana school 
for Mathematics, Science, and the Arts distance education program has 
been in operation for 7 years. An incomplete evaluation conducted in 
1990-91 was compared to a completed 1991-92 evaluation to demonstrate 
that evaluations of the same program from different approaches can 
yield similar results. In 1990-91, 81 schools received 10 subjects, 
and in 1991-92, 113 schools received 13 high school courses, serving 
942 students. The program uses audiographics, a distance education 
technology that includes a personal computer, an electronic penpad, 
an audio convener, a modem, and a high resolution monitor. The 

1990- 91 study was student-focused and student-outcome oriented. The 

1991- 92 study was teacher-focused and process oriented. The 1990-91 
evaluation focused on the effectiveness of the project as perceived 
by students, who evaluated the delivery system, p^^rceDtions of course 
quality, and teacher effectiveness. The 1991-92 stady focused on 15 
full-time and 8 part-time teachers and the teaching processes. Both 
studies used ^ student attitudinal survey, incorporated on-site 
student interviews, and emphasized the key role of local proctors. In 
the second evaluation, teachers were also interviewed. Both studies 
indicf>to that students were very positive about the system and 
comfortable with the technology. Student outcomes and teacher 
processes highlight the Telelearning project.' s advantages. (SLD) 



ED 355 934 

AUTHOR 
TITLE 
PUB DATE 
NOTE 
PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



* Reproductions supplied by EDRS are the best that can be made * 

from the original document. * 



TELELEARNING: A SECOND LOOK 



1990 - 1991 

1991 . 1992 



U.S. DEPARTMENT OF EDUCATION 
Oft»ce of EducAtionat Resoarch and improvement 

EDUCATIONAL aESaUPCES INFORMATION 
CENliw^ERlC) 

C This document has been reproduced as 
recejved from the person or organization 
originating it 

C Minor Changes have been made to improve 
reproduction Quality 



• Points of view or opinions stated m this docu- 
ment do noi necessarily represent official 
OERl position or policy 



Lee IC McEIveen, Louisiana Evaluation Enterprise, INC 
Sharon P. Roberts, Louisiana State University 



September 25, 1992 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 

Sharon RoT^f^-tg 



BEST COPY AV/ULIBLE 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



1 

L TELELEARNING - A SECOND LOOK 

Although Teleleaming has been operational in the State of Louisiana for seven years, the 
program has been fonnally evaluated only during the last two school years. The first evaluation 
coinr>issioned for the 1990-91 school year was not completed by the original contract evaluator. The 
second formal evaluation was done for the school year 1991-92. A comparison and contrast of the 
two evaluations contains a ironic twist. Even though the evaluations looked at different objectives 
using different designs and different methods, the results and recommendations were similar. 

Both of the evaluations were conducted to " determine the degree of effectiveness of the 
Statewide Distance Learning Program - Teleleaming." Even though they both had the same mission, 
the objectives of each evaluation were different The focus of the first evaluation was student- 
oriented. The three goals dealt mainly with ttie students' acceptance, bvel of perforaiance and extent 
of enhancement Telelearning had on producing students eligible for Louisiana scholarships and 
college. The focus of the second evaluation was teacher-oriented. The three goals were assessed by 
using both student and teacher perceptions of the effectiveness of the Teleleaming delivery system, 
quality of the Teleleaming courses as delivered by the teachers and the effectiveness of the teachers. 

This paper will look at the similarities and differences in the two evaluations. The purpose of 
this paper is to validate the fact that two evaluations of the same program, using different approaches 
can yield the same results. The idea for the paper came about when one evaluator, who worked on 
both projects, put the recommendations side by side and realized that the same findings occurred 
virtually in both studies. 

This paper is divided into three sections. The first section describes the Teleleaming project; 
the second section describes the 1990-91 evaluation design and the 1991-92 evaluation design and then 
contrasts and compares the two designs; the third section describes the conclusions and 
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recommendation of the 1990-91 evaluation and the 1991-92 conclusions and recommendation and then 
contrasts and compares the two findings, 
n. BACKGROUND RELATED TO TELELEARNING 
A. Description of the Telelearning Program 

Teleleaming, Project Outreach, a Louisiana School for Math, Science and the Arts distance 
education program, is in its seventh year. In 1986, a two school network was created to test the 
feasibility of teaching Calculus and Survey of the Arts to schools using audiographics technology. 
The State Department of Education in conjunction with the Board of Elementary and Secondary 
Education with 8g Quality Education funds approved an expansion of the system to reach as many as 
114 rural schools in Louisiana. For 1990-91 the actual network size was 81 schools receiving ten 
subjects (French I, II, III, Spanish I, II, III, Advanced Mathematics, Calculus, Physics and Survey of 
the AnsO Courses were taught by seven (7) full-time and eight (8) adjunct teachers. Most of the 
participating schools offer two courses through the Teleleaming network. Thirteen courses were 
offered through tiiis program during the 1991-92 school year to 113 schools within 44 school districts, 
serving 942. The additional courses were German I, Latin I, and Computer Science. Courses were 
taught by fifteen (15) full-time and eight (8) part-time teachers. 

This program utilizes audiographics, a unique distance education technology that includes a 
personal computer, electronic penpad, audio convener, modem and high resolution monitor. The 
system is a computer based, interactive audio and graphics system that incorporates graphics and 
single frame video images stored in and utilized by a special patented software package that operates 
the program and its peripherals. In addition, a common telephone line is used to deliver the audio 
component of the system and digitized computer information. The PC controls the system through a 
modem that separates speech from the computer data. The teacher communicates with the class via an 
audio convener that processes tiie sound by sending tiie teacher's conversation or student's responses 
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via the phone line. The teacher has a bank of computer images stored in a special directory and 
created with special graphics package stored in the operating soft^yare. The software also includes a 
video component that allows the user to capture and store any video image provided from a NTSC 
video signal (ex.: video camera, VCR, video disk). Besides the keyboard, a second peripheral is the 
penpad. This component serves as an "electronic chalkboard" to write or receive written information 
between the participating schools. The penpad is also the source device for creating graphics used by 
the teacher for instruction. 

A typical Teleleaming class includes four schools plus the teaching site with five to seven 
students per school. This duplicates a regular classroom in size. The students receive instruction via 
computer three days per week aiid work independently the other two days. Each station has up to five 
microphones to provide voice contact between teacher and students. The system encourages 
interaction between the teacher and students. Each participating school provider a proctor to facilitate 
the class by taking roll and administering materials, 
assignments and exams. 

m. DESCRIPTION OF THE 1990-91 EVALUATION STUDY 
A. Purposes of the Evaluation 

The puipjo'^e of this project was to evaluate student acceptance, student performance, and 
curriculum enhancement of the Louisiana School Distance Learning Program as it delivers certain 
courses (required by the Taylor Scholarship Plan [TAP] and for admission to Louisiana State 
University) to small and remote rural schools throughout Louisiana. 

In order to determine the relative success of Teleleaming in promoting student learning, the 
enhancement of rural high schools in Louisiana and the student preparation for scholarships-a variety 
of strategies were applied to this study. Telephone interviews, survey infomiation, demographic 
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information, on-site interviews, LSU bridge observations and grades were employed to identify the 
Teleleaming user. 

B. Evaluation Objectives for 1990-91 

The objectives for this project as derived from the evaluability assessment are: 

1. To assess the degree of acceptance of the Teleleaming program by 
participating students. 

2. To detennine the level of performance of students in various courses offered 
through the Teleleaming Program. 

3. To assess the extent of enhancement the Teleleaming program has had on 
producing students eligible for Louisiana scholarship programs and college 
entrance. 

C. Evaluation Design for 1990-91 

At the onset of the evaluation on January 25, 1991, the coordinators of the evaluation met with 
the evaluation team to explain the purpose of the evaluation and to solicit their support on the project 

Telephone interviews were conducted with the project director and the funding agency 
manager. Visits were then made to Louisiana School for Math, Science and Aits for personal 
interviews w.th project director. Sampling procedures were decided, the questionnaire was designed 
and the mid-temi grades were requested. Visits to 16 sample schools were made for class observations 
and interviews with students, proctors and principals. Schools that could not be visited in person were 
"observed" at the LSU bridge site, which is actually a Baton Rouge Teleleaming station. Nine classes 
were observed for thirty minutes in each of the three content areas. Also members of the evaluation 
team observed different schools on different days while different subjects were being taught, in order 
to col 'eci uaia from many varied situations, circumstances and per eptions of Teleleaming users. 

Another method of collecting data was through responses to the student attitudinal survey. It 
identified student satisfaction in terms of delivery mechanism, convenience of use, time effectiveness 
of the system, perceptions regarding the instmctor, communication patterns (interactions between 
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students and teachers) and the effects of having a proctor, parent or teacher in the class. Interviews 
with students gave additional indication of student's level of acceptance. 

Mid-term and final grades were collected and used to assess student performance. Grades 
were slow being reported and were not systematically recorded. In fact, phone calls to the 
Teleleaming office were made in order to have missing grades sent to the evaluation team. Missing 
grades were extremely valuable in that they identified program drop-outs. 

The three content areas were investigated separately: a) Foreign Languages Courses; b) Math 
and Physics Courses; and c) Arts Courses. For each of these three areas a total of nine samples of 
thirty minutes each were observed. Three of these were on-site and six were randomly selected such 
that the six sessions did not cover the same teacher, same topic, or the same time of day. The three 
on-site sessions were in three different cities. The locations were selected at random firom the sixteen 
sample sites. The twenty-seven observations lasted tiirough May 17th, the last day of the Teleleaming 
broadcast. Student profile information was collected to identify students, courses, demographic 
information, sex, classification, schools, parishes, teachers, age and mid-tenm grades. This data was 
entered into a computer program so that matching was accomplished easily. Final grades and college 
plans were collected at the end of the school year. 

During late May and early June, telephone interviews were conducted to obtain essentially the 
same information from studer.*s who dropped out of the program and were iiot reached through the 
written surveys. Non-respondents were also contacted through this medium. 

In order to assess the effectiveness of the Teleleaming Program in producing students eligible 
for Louisiana scholarship programs, information about student records of grades, college applications, 
scholarship applications, and the survey results were utilized. This process produced a Teleleaming 
student profile which indicated those students who are eligible for Louisiana Scholarship Programs 
according to the set standards. Guidance counselors at the participating schools were valuable contacts 
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for infonnation about students receiving scholarships. The survey identified student satisfaction in 
tenns of delivery mechanism, convenience of use, time effectiveness of the system, perceptions 
regarding instructor, communication patterns (interactions between students and teachers), and the 
effects of having a proctor, parent, or teacher in the class. Observations and informal interviews with 
students during on-site visits gave an additional indication of student's level of acceptance. Drop-outs 
and non-respondents contributed vital infonnation to this process, 
IV, DESCRIPTION OF THE 1991-92 EVALUATION STUDY 

A. Purposes of the Evaluation 

The purpose of the evaluation was to detennine the degree of effectiveness of the Teleleaming 
Project by focusing on studencs' perceptions of the actual delivery system, students* perceptions of 
course quality as delivered through the system and Teleleaming teacher effectiveness. 

B. Evaluation Objectives for 1991-92 

The following objectives were based on the determination that student perceptions of the 
system itself, the quality of courses offered and teacher effectiveness had not been extensively 
explored and needed to be addressed: 

1. To assess student perceptions of tlie effectiveness of the Teleleaming delivery system. 

2. To assess student perceptions of the quality of the Teleleaming courses 

3. To assess the effectiveness of the teachers' delivery of the courses. 

C. Evaluation Design for 1991-92 

Meetings were conducted with Louisiana State Department of Education program directors to 
finalize agreement on investigatory procedures. A combination of quantitative and qualitative 
approaches were used to evaluate the program. The designed consisted of: (1) student attitudinal 
survey, (2) on-site student interviews, (3) teacher observations via a telecommunications bridge, (4) 
on-site teacher interviews and (5) comparison of student achievement and attitude. 

8 
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The student attitudinal instrument developed by the program evaluators used on previous 
student surveys sent out by LSMSA's Teleleaming project in an effort to demonstrate continuity " in 
the eyes of participants schools" and to assure LSMSA Teleleaming faculty and staff of the evaluation 
teams's desire to work cooperatively with them for the purpose of program involvement 

On site one-on-one interviews were conducted by a member of the evaluation team at 
randomly selected schools sites. A total of fifty-nine students were interviewed in eight of the forty- 
four member parishes. The evaluation team member conducted interviews on-line and off-line days 
and therefore had an opportunity to witness class activities under a variety of circumstances. The 
student interview instrument mirrored several of the questions posed in the Student Teleleaming 
Survey and were intended to probe for in-depth responses. 

The Teleleaming teacher's ability to effectively convey course content through the 
Teleleaming system is crucial to its success. An observation instrument was developed to observe 
individual teachers instruction by means of a telecommunications bridge simultaneously linking 
program observers with multi-site class sessions. Thirteen(13) three member teams, each composed of: 
(10 a pedagogical observer, (2) a subject-specific observer and (3) a generalist observer well versed in 
oral communications observed thirty-nine (39) fifty- minute individual observations in total. 

Interviews with all ftill-time and part-time Teleleaming teachers were conducted at the 
Louisiana School for Math, Science and the Arts. Teachers were dickcd to respond to thirteen 
questions categorized as eight open-ended and five *'yes/no" response-type questions. Teachers were 
asked to supply numerical ratings on a 1-10 scale to rate the current Teleleaming system and to rate 
the proctor's importance within the instmctional structure. 

Teleleaming teachers' written self-evaluations were also provided unexpectedly to the 
evaluator during the visit. The self-evaluations provided additional qualitative data which lended to 
the overall impression of the program, but were not used in the analysis of the program. 
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Program evaluators requested and were provided witii a list of grades for all students by 
LSMSA. Of the 813 completed and returned Student Attitudinal Surveys, 799 were matched with 
final grades. A statistical analysis was performed to determine the relationship of student grades and 
student attitudes. 

V. COMPARISON AND CONTRAST OF THE DESIGN OF THE TWO TELELEARNING 
STUDIES 

Interestingly enough, although the findings of both evaluadons studies were very similar, the 
focus, objectives and evaluation procedures contrasted in several ways. The 1990-91 study was 
student-focused and student-outcome oriented, while the 1991-92 study was teacher-focused and 
process oriented. 

Both studies incorporated a student attitudinal survey. The 1990-91 used the LSMSA survey 
with additions, the 1991-92 used an original survey in similar design which included a few of the 
LSMSA survey items. The procedure for dispersement and collection of surveys was also dissimilar 
in that the 1990-91 study sent our and received all surveys through LSMSA. The 1991-92 survey was 
sent out and returned directly to program evaluators. 

Both studies incorporated on-site student interviews. The 1990-91 study concentrated in one 
central location; the 1991-92 study used a geographical spread. Similarly, the evaluation team member 
interviewed during on-line and ofMine days, fomally and informally, in bcth studies. Interview focus 
was dissimilar to great extent, in that the 1990-91 study was probing for further "student opinion, 
goals and grade" data and the 1991-92 study was probing further "student opinion of the delivery 
system and teacher delivery" data. Student access problems were the same for both studies and 
involved careful scheduling of interview appointments to insure data collection, which included a 
cross-section of student courses, course delivery times and a variety of classroom situations. 
Consideration for regular student schedules, testing, school holidays, etc. were necessary for successful 
implementation of both studies. 
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The 1990-91 study continued to focus on student outcome by determining correlations between 
student attitude and achievement by content areas and further probing for scholarship eligibility data as 
a result of taking Teleleaming courses. The 1991-92 study compared student attitude and achievement 
by individual courses, but did not investigate beyond that. 

Similarly, both studies used the LSU telecommtmication bridge site to "observe" on-line 
classes. The focus for tiie 1990-91 study was student-teacher interaction; the focus for tiie 1991-92 
study was teacher performance and delivery of the curriculum through the system. 

Direct involvement witii LSMS A teachers was limited to an on-site observation at the LSMSA 
by evaluation team members for tiie 1990-91 study. Dissimilarly, tiie teacher-focused 1991-92 study 
included on-site observations and in-depth one-on-one interviews of program teachers, faculty and 
staff. 

VI. CONCLUSIONS AND RECOMMENDATIONS OF THE 1990-91 EVALUATION 
A. Conclusions of the Evaluation 

This project set out to evaluate three areas of Teleleaming: student acceptance of Teleleaming, 
student performance in Teleleaming classes, and curriculum enhancement necessary for college bound 
students. The report contains quantitative and qualitative analysis of data to deteraiine the 
effectiveness of the program. 

The degree of acceptance of the Teleleaming program by participating students was high. The 
level of perfomiance of students in various courses offered through the Teleleaming program was 
above average. The extent of enhancement the Teleleaming program had on producing students 
eligible for Louisiana scholarships and college entrance was also significant. 

According to the questionnaire and interviews most students were satisfied witii Teleleaming 
distance education as a means of receiving instruction. Student recmitment, types of proctore, course 
format and late start-up were major influences toward student attitude. Prior to the evaluation, the 
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general thought was that the Survey of the Arts course would lend itself to the Teleleaming system 
with greater student satisfaction. The Survey of the Arts class is mainly visual, whereas foreign 
language is largely oral in format. The math courses are a combination of both oral and visual 
learning. 

The findings of this evaluation showed that student satisfaction with Teleleaming was greater 
in the math and foreign language areas, not the Survey of the Arts as had been anticipated. These 
findings, however, do not suggest that Survey of the Arts is not appropriate for the Teleleaming 
system. The Survey students were satisfied, just not as much as the language and math students. 
There were many qualified reasons for the lesser success m6 satisfaction of the Survey students. 
Student dissatisfaction resulted from students being "forced" to take the Teleleaming course or the way 
in which the course was presented. The Pearson Correlation Coefficient indicated extremely high 
correlations between student attitudes toward teacher, equipment/delivery system and the 
courses/subject/class with satisfaction being the highest in the course category. Negative comments 
also came from students who did not have trained proctors in the Teleleaming Classes. Students not 
being monitored were not encouraged to take the courses seriously or to be conscientious students. 
The math subject group reported the most positive attitude and greatest satisfacdon with Teleleaming. 

Students enjoyed the class meeting only three days a week and having two days to complete 
assignriients. The equipment and delivery system seemed to fascinate students, as today's students are 
geared toward computers. 

Another important aspect of the program concerned student perfomniance. The students who 
completed Teleleaming courses generally had above average grades. Those students who did not 
perfomi well either were not prepared for the course, did not want to be in the class or did not have a 
class proctor Again, it was not the Teleleaming program which prevented grades ftom being high. 
The math subject areas had the highest grades, indicating courses which were more adaptable and 
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appropriate to Teleleaming as a means of instructing. The students in the other subject areas, foreign 
language and survey of the arts, similarly had high grades. With proper teacher training in the use of 
the Teleleaming system the courses can be more "natural" for students, that is, more like being in a 
traditional class. An important factor accounting for such overall high grades was that a large 
number of students with low grades dropped out of the program and the majority of these students 
were in the Survey of the Arts class. The average grades would have been much lower had these 
students remained in the program. 

Teleleaming did produce some students who met Louisiana college entrance requirements and 
who were qualified for Louisiana scholarships. The questionnaire identified 207 students who were 
taking Teleleaming courses for these reasons. However, in the interviews it became apparent tliat 
many students were not aware of such scholarships or the Taylor Plan. Student recruitment then 
became a key issue. The Louisiana State Department of Education identified twice the number of 
students in Louisiana who needed certain courses to be eligible for college entrance or Louisiana 
Scholarships as were enrolled in the Teleleaming program during 1990-1991 Finally, the program's 
greatest strengths and successes concerned the potential impact of Teleleaming. Because of . urriculum 
enhancements courses necessary for students to be eligible for college entrance or a scholarship were 
available at no additional fee. 

This novel teaching/leaming system provides high quality instmction to mral, academically 
deprived students in Louisiana who might not have courses available to them. 

Thus, the Teleleaming experience can change the entire future for the students who take 
advantage of this educational opportunity, 

B. Recommendations of the 1990-91 Evaluation 

Quantitative data were obtained ft^om the student survey and grades. Qualitative data came 
from interviews, observations, and the stuaent questionnaire. 

o 13 
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Several recommendations emerged from the evaluation. 

1. Continue to use the Louisiana School as the distributor of Teleleaming. The staff and 
faculty are dedicated to producing a successful program. 

2. Recruit students that need Uiis program for college and scholarship purposes. The 
primary reason the program was funded was to encourage more Louisiana students to 
attend college. 

3. Teleleaming equipment should be used for pre-service and in-service during off-days. 
Additional uses of tiie equipment, such as for professional development, should be 
permitted. 

4. The Teleleaming teachers and proctors should be trained in using the equipment, 
capabilities and techniques of distance education. This would increase effectiveness of 
this method of instruction. 

5. Schedule and plan die program so that all classes start up at tiie same time. Adopt a 
Teleleaming calendar for participants to follow and post before start-up. 

6. Coordinate and present each course in a logical manner. Class sessions must be 
organized and follow a logical and long range plan. 

7. Locate classes in a space that is large enough for the equipment and the students. 
Proper facilities and location are essential and conducive to the Teleleaming success. 

8. Provide compatible software and working equipment for each class. Qeai' telephone 
lines and quick repair/replacement of equipment are critical. 

9. Administer evaluation questionnaires earlier and provide make up classes to make 
more effective use of time. 

10. Computerize records and grades. Accuracy of infonnation and easy access to the 
information is needed. 



Vn, CONCLUSIONS AND RECOMMENDATIONS OF THE 1991-92 EVALUATION 
A. Conclusions of the Evaluation 



This evaluation of the Teleleaming Project addressed three specific issues: 

Student perceptions of the effectiveness cf the Teleleaming delivery system. 
Student perceptions of the quality of the Teleleaming courses. 
Effectiveness of the teachers delivering the Teleleaming courses. 
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Evidence gathered by means of a written survey instrument, on-site interviews, teaching 
observations and final grade analyses served to produce the findings presented below. 

B. Student Perceptions of the Effectiveness of the Telelearning Delivery System 
Students were very positive about learning by means of the Telelearning system. They were 

pleased to have the opportunity to take courses which were previously unavailable to them, they were 
intrigued by the technology which provided a change from the traditional classroom situation, and the 
majority of them enjoyed being-on-line with students firom other areas of the state as long as they were 
"compatible". 

Most students were comfortable with the technology and handled equipment with ease. 
Technical problems affecting the effectiveness of the system were predominately caused by "pulled" 
wires on school site-microphones and telephone line static. Enhancement of communication skills was 
a natural by-product of this delivery system. Final grade averages for all courses reflected average- 
and-above scores for all courses with a statistically significant relationship between grades and 
attitudes noted. 

Result of the analyses of all evaluation instruments indicated a very positive perception of the 
effectiveness of the Telelearning delivery system by students. 

C. Student Perceptions of the Quality of the Telelearning Courses 

Results of tlie analyses of the attitudinal survey instrument and on-site student interviews 
indicated that students were pleased with the quality of the courses they were receiving through 
Telelearning. They felt they were receiving "college-level" instniction from "college-level" teachers. 
Students in foreign language courses generally indicated frustration because of the lack of a visual 
element to the system - a particularly important element in teaching a language. A positive response 
of 80.7% by students indicated that they learned more in their Telelearning class than they would have 
in a traditional class. The Monday/Wednesday/Friday on-line schedule with Tuesday/Thursday 
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scheduled off-line days, fostered independence, with more responsibility being placed on the student to 
"get the job don". For that reason, the maturity level of the student should be one of the major 
determinants for enrollment in Teleleaming courses. 

D. Effectiveness of the Teachers Delivering the Telelearning Courses 

Results of teachers observations and interviews were generally positive. Teachers were 
knowledgeable in their respective areas and were generally able to convey that knowledge to students 
via audiographics. 

A lack of personal contact and visual feedback were cited by Teleleaming teachers and 
students alike as being the major drawbacks of this type of system. Teachers generally connected the 
"personal contact issue" with their effectiveness as teachers. They also express concern regaining 
timely feedback to students and acknowledged without reservation that the classroom proctor was a 
major component of the effectiveness fonnula. 

E. Recommendations of the 1991-1992 Evaluation 

Consideration should be given to the following points as recommendations for the purpose of 
maintenance and improvement of the quality of the Teleleaming Program and are based on the 
findings of the evaluation. 

Special attention should be given by those audiences identified at the end of each 
recommendation. 

Audience Identification Key 
TD = Teleleaming Director 
TF = Teleleaming Faculty 
SD = State Department Staff 

1. Provide a summary of course descriptions to students prior to enrollment so that they 
understand the scope of the course. (TD,TF) 

2. Avoid excessive coverage of content paiticulariy in lecture-type classes. (TF) 
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3* Establish a visual cue for students's questions to teachers for the purpose of more 
effective communication. (TD,TF) 

4. Increase the use of visuals during on-line instructional time to maintain student 
interest. (TF) 

5* Revitalize the 1-800 "homework line" for use as a student resource. (TD,TF) 

6. Develop a consistent grading criteria among Survey of the Arts teachers across all 
courses within that area. (TF) 

?• Coordinate the teaching of "eras" among Survey of the Arts courses to allow for easier 
transition from one course to the other and to provide overall continuity to the course. 
(TF) 

8» Increase effective faculty commimication by providing regularly scheduled 
meetings.(TD) 

9* Provide formal technical training for new teachers to the Teleleaming system. (TD) 

10. Provide training in teaching methodology for new teachers to the Teleleaming system. 
(TD) 

11. Expand the use of the Teleleaming system by providing inservices, workshops, etc. for 
the purpose of professional development (SD,TD) 

12. Provide timely notification of all Teleleaming events to appropriate State Department 
Staff. (TD) 

13. Visit Teleleaming school sites to conduct on-line classes, particularly those schools 
considered to have instmction-related problems. (TF) 

14. Provide a Spring event for students as a follow-up to the Fall orientation. (TD,TF) 

15. Provide faculty and class pictures on screen and in print to aU Teleleaming classes. 
(TD,TF) 

16. Study the role of the proctor more closely because of its key position within the 
Teleleaming structure(SD) 
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Vm. CONTRAST AND COMPARISON OF THE 1990-91 AND 1991-92 EVALUATION 
RESULTS AND RECOMMENDATIONS 

Although objectives were dissimilar and different implementation procedures were employed, 

both studies concluded with similar findings. For both studies: 

1. There was a high correlation between student achievement and attitude. 

2. Student opinions of the system were similar. 

3. Classroom proctors were found to be an extremely important component of the system, 
at times, being the crucial determinant of student success or failure of the course. 

4. Foreign language courses were found to be the most difficult to teach through the 
system because of lack of visual feedback. 

Again, recommendations for the 1990-91 and the 1991-92 studies parallelled. Similar 
recommendations included the following: 

1. Provide inservice training for teachers and proctors in the use of equipment and 
method of instruction through this type of system. 

2. Expand the use of the system for professional development during off-line days. 

3. Develop course continuity through coordinated planning efforts by the faculty. 

4. Due to the nature of the medium, local schools should strongly consider student 
preparation and maturity level when scheduling Teleleaming courses, particularly 
foreign languages. 

5. Provide a spring event for students as a foUovv-up to the Fall orientation. 

6. Increase the use of visuals during on-line instruction time to maintain student interest. 

7. Revitalize the 1-800 homework line for use as a student resource. 
IX. SUMMARY 

The purpose of both the 1990-91 informal study and the 1991-92 formal study was to provide 
an overall evaluation of the Louisiana Teleleaming Program. Although the focus of each study was 

i8 
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different, the results were very similar,, providing multiple confirmation of findings. Similarities 
included: 

1. Students were very positive about learning by means of the Teleleaming system. 

2. Students were comfortable with the technology. 

3. Technical problems were predominately related to minor microphone wiring difficulties 
at local school sites and telephone line static during inclement weather. 

4. Students enjoyed the independent college-type atmosphere cr&ated by having on-line 
instruction and off-line independent study. 

5. Student level of maturity was a major determinant of successful course completion. 

6. Proctors played a key role in successful program implementation. 

The previous list is a fairly comprehensive evaluation of the Teleleaming program and all of 
these results were found in both evaluations. The 1991-92 evaluation was more in-depth compared to 
the 1990-91. The 1991-92 evaluation had a response rate of 98% on surveys and was more global in 
scope of interviews. All teachers were formally interviewed and a wider area of the state was covered 
in the student interviews in 1991-92. The contrast is important, but the interesting aspect is that the 
findings so closely paralleled each other. Many of the recommendations of the 1990-91 study were 
written in narrative fonn and expressed the same suggestions that the 1991-92 recommendations 
numerically Usted. 

The findings of the 1990-91 study were not fonmally presented to the State funding agency nor 
the LSMSA. Therefore, the agency nor the school benefitted from the complete findings. Some of 
the results were made known to the LSMSA via telephone conversations. It is evident that the 
suggestion to computerize the record system was implemented for the 1991-92 school year, because 
the evaluation for that year used the computerized records to track surveys. The next evaluation of the 
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Teleleaming program should provide insight into whether the recommendations made by the 1991-92 
study will be implemented. 

Whether looking at student outcomes or teacher processes, the findings remain the same for 
this educational program. An overall description of the programs strengths an we^Jcnesses can 
sometimes surface from an evaluation no matter the method or the design. 
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